History-based Feature Selection (HBFS) is a feature selection tool that aims to identify an optimal subset of features for prediction problems. It is designed to work similarly to wrapper methods and genetic methods, focusing on selecting feature subsets that yield the highest performance for a given dataset and target. HBFS differs from filter methods, which evaluate and rank individual features based on their predictive power. Instead, HBFS evaluates combinations of features over multiple iterations, using a Random Forest regressor to estimate performance and iteratively refining feature sets. This tool supports binary and multiclass classification, as well as regression, and allows for balancing the trade-off between maximizing accuracy and minimizing the number of features through parameters such as maximum features and penalties. Examples provided demonstrate the use of HBFS with various models and metrics, showcasing its ability to improve model performance by identifying optimal feature subsets.
The article discusses the credibility of using Random Forest Variable Importance for identifying causal links in data where the output is binary. It contrasts this method with fitting a Logistic Regression model and examining its coefficients. The discussion highlights the challenges of extracting causality from observational data without controlled experiments, emphasizing the importance of domain knowledge and the use of partial dependence plots for interpreting model results.
- Extreme Gradient Boosting: A quick and reliable regressor and classifier
- Summary: LightGBM is faster and better though XGBoost is close
- Summary: LightGBM is faster and better though XGBoost is close
In this post, I’ll discuss random forests, another popular approach for feature ranking.
Random forest feature importance
Random forests are among the most popular machine learning methods thanks to their relatively good accuracy, robustness and ease of use. They also provide two straightforward methods for feature selection: mean decrease impurity and mean decrease accuracy.
Random forest feature importance
Random forests are among the most popular machine learning methods thanks to their relatively good accuracy, robustness and ease of use. They also provide two straightforward methods for feature selection: mean decrease impurity and mean decrease accuracy.
